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Abstract. In previous work we showed that two apparently unrelated formu- 
las for the Hall-Littlewood polynomials of type A are, in fact, closely related. 
The first is the tableau formula obtained by specializing g = in the Haglund- 
Haiman-Loehr formula for Macdonald polynomials. The second is the type 
A instance of Schwer's formula (rephrased and rederived by Ram) for Hall- 
Littlewood polynomials of arbitrary finite type; Schwer's formula is in terms 
of so-called alcove walks, which originate in the work of Gaussent-Littelmann 
and of the author with Postnikov on discrete counterparts to the Littelmann 
path model. We showed that the tableau formula follows by "compressing" 
Ram's version of Schwer's formula. In this paper, we derive tableau formu- 
las for the Hall-Littlewood polynomials of type B and C by compressing the 
corresponding instances of Schwer's formula. 



1. Introduction 

Hall-Littlewood polynomials are at the center of many recent developments in 
representation theory and algebraic combinatorics. They were originally defined in 
type A, as a basis for the algebra of symmetric functions depending on a parameter 
t; this basis interpolates between two fundamental bases: the one of Schur functions, 
at t = 0, and the one of monomial functions, ait = 1. Beside the original motivation 
for defining Hall-Littlewood polynomials which comes from the Hall algebra [18], 
there are many other applications (see e.g. [12^ and the references therein). 

Macdonald [20j showed that there is a formula for the spherical functions corre- 
sponding to a Chevalley group over a p-adic field which generalizes the formula for 
the Hall-Littlewood polynomials. Thus, the Macdonald spherical functions general- 
ize the Hall-Littlewood polynomials to all root systems, and the two names are used 
interchangeably in the literature. There are two families of Hall-Littlewood polyno- 
mials of arbitrary type, called P-polynomials and Q-polynomials, which form dual 
bases for the Weyl group invariants. The P-polynomials specialize to the Weyl char- 
acters at i = 0. The transition matrix between Weyl characters and P-polynomials 
is given by Lusztig's t-analog of weight multiplicities (Kostka-Foulkes polynomials 
of arbitrary type), which are certain afhne Kazhdan-Lusztig polynomials [9l I19j. 
On the combinatorial side, we have the Lascoux-Schiitzenberger formula for the 
Kostka-Foulkes polynomials in type A [TD] , but no generalization of this formula to 
other types is known. Other applications of the type A Hall-Littlewood polynomials 
that extend to arbitrary type are those related to fermionic multiplicity formulas 
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[2] and afRne crystals ^ . We refer to [231 El] for surveys on Hall-Littlewood 
polynomials of arbitrary type. 

Macdonald [HI [52] defined a remarkable family of orthogonal polynomials de- 
pending on parameters q, t, which bear his name. These polynomials generalize the 
spherical functions for a p-adic group, the Jack polynomials, and the zonal poly- 
nomials. At q = 0, the Macdonald polynomials specialize to the Hall-Littlewood 
polynomials, and thus they further specialize to the Weyl characters (upon setting 
t — Q as well). There has been considerable interest recently in the combinatorics 
of Macdonald polynomials. This stems in part from a combinatorial formula for 
the ones corresponding to type A, which is due to Haglund, Haiman, and Lochr 6]. 
This formula is in terms of fillings of Young diagrams, and uses two statistics, called 
inv and maj, on such fillings. The Haglund-Haiman-Loehr formula already found 
important applications, such as new proofs of the positivity theorem for Macdonald 
polynomials, which states that the two-parameter Kostka-Foulkes polynomials have 
nonnegative integer coefficients. One of the mentioned proofs, due to Grojnowski 
and Haiman [5j, is based on Hecke algebras, while the other, due to Assaf [3|, is 
purely combinatorial and leads to a positive formula for the two-parameter Kostka- 
Foulkes polynomials. Moreover, in the one-parameter case (i.e., when q = 0), 
the Haglund-Haiman-Loehr formula was used to give a concise derivation of the 
Lascoux-Schiitzenberger formula for the Kostka-Foulkes polynomials of type A [SI 
Section 7]. 

An apparently unrelated development, at the level of arbitrary finite root sys- 
tems, led to Schwer's formula [26], rephrased and rederived by Ram [24], for the 
Hall-Littlewood polynomials of arbitrary type. The latter formulas are in terms of 
so-called alcove walks, which originate in the work of Gaussent-Littelmann [?] and 
of the author with Postnikov [Ml [15] on discrete counterparts to the Littelmann 
path model [16|,ll7j. Schwer's formula was recently generalized by Ram and Yip to 
a similar formula for the Macdonald polynomials ^25]. The generalization consists 
in the fact that the latter formula is in terms of alcove walks with both "positive" 
and "negative" foldings, whereas in the former only "positive" foldings appear. 

In [12], we related Schwer's formula to the Haglund-Haiman-Loehr formula. 
More precisely, we showed that we can group the terms in the type A instance 
of Schwer's formula (in fact, we used Ram's version of it) for P\{x] t) into equiva- 
lence classes, such that the sum in each equivalence class is a term in the Haglund- 
Haiman-Loehr formula for q = 0. An equivalence class consists of all the terms 
corresponding to alcove walks that produce the same filling of a Young diagram A 
(indexing the Hall-Littlewood polynomial) via a simple construction. In fact, we 
required that the partition A has no two parts identical (i.e., it is a regular weight); 
the general case, which displays additional complexity, will be considered in a future 
publication. This work was extended in |13j . by showing that the type A instance 
of the Ram- Yip formula for Macdonald polynomials compresses, in a similar way, 
to a formula which is analogous to the Haglund-Haiman-Loehr one, but has fewer 
terms. 

In this paper we extend the results in [12] to types B and C. More precisely, 
we derive formulas for the Hall-Littlewood polynomials of type B and C indexed 
by regular weights in terms of fillings of Young diagrams; we do this by compress- 
ing the corresponding instances of Schwer's formula (in fact, we again use Ram's 
version of it). Note that no tableau formula for the Hall-Littlewood or Macdonald 
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polynomials exists beyond type A so far. Our approach provides a natural way 
to obtain such formulas, and suggests that this method could be further extended 
to type D (this case is slightly more complex than types B and C, see below), as 
well as to Macdonald polynomials; these problems are currently explored, as is the 
compression in the case of a Hall-Littlewood polynomial indexed by a non-regular 
weight. Our formula is more complex than the corresponding one in type A (i.e., 
the Haglund-Haiman-Loehr formula at q = 0); however, the statistic we use is, in 
the case of some special fillings, completely similar to the Haglund-Haiman-Loehr 
inversion statistic (which is the more intricate of their two statistics). The natu- 
rality of our formula is also supported by the fact that the Kashiwara-Nakashima 
tableaux of type B and C [8j are, essentially, the surviving fillings in this formula 
when we set t — 0. We also note that that the passage from (Ram's version of) 
Schwer's formula to ours results in a considerably larger reduction in the number 
of terms in type B and C compared to type A. In terms of applications, it would 
be very interesting to see whether our formula could be used to derive, in the spirit 
of [H Section 7], a positive combinatorial formula for Lusztig's t-analog of weight 
multiplicities in type B and C, which has been long sought. 

2. The tableau formula in type C 

Let us start by recalling the Weyl group of type B/C, viewed as the group of 
signed permutations _B„. Such permutations are bijections w from [n] := {1 < 2 < 
. . . < n < n < n — 1 < . . . < 1} to [n] satisfying w(7) = w{i). Here 7 is viewed as 
— i, so I — i. We use the window notation w = w{l) . . . w{n). Given 1 < i < j < n, 
we denote by the reflection which transposes the entries in positions i and 

j (upon right multiplication). Similarly, we denote by (i,j), again for i < j, the 
transposition of entries in positions i and j followed by the sign change of those 
entries. Finally, we denote by («,7) the sign change in position i. Given w in i?„, 
we define 

(2.1) £+{w) := \{{k,l) : 1 < k < I < n, w{k) > w{l)}\ , 

£_(w) \{{k, I) : l<k<l<n, w{k) > ^I)}| . 

Then the length of w is given by l{w) := £+(w) -I- £-{w). 

Let A be a partition corresponding to a regular weight in type Cn for n > 2, 
that is A = (Ai > A2 > . . . > A„ > 0). Consider the shape A obtained from A by 
replacing each column of height k with kov2k — l (adjacent) copies of it, depending 
on the given column being the first one or not. We are representing a filling cr of A 
as a concatenation of columns Cij and C'^^., where i — 1, . . . , Ai, while for a given 
i we have j = 1, . . . , A^ if i > 1, j = 1 if « = 1, and fc = 2, . . . , A^; the columns Cij 
and C[f. have height A'^. More precisely, we let 

(2.2) CT = C^^...CS 
where 

Note that the leftmost column is Cai,i, and the rightmost column is Cn. 

Essentially, the above description says that the column to the right of Cij is 
Cij+i, whereas the column to the right of C[f. is C- ^.^t^. Here we are assuming that 
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the mentioned columns exist, up to the following conventions: 

„s ^ f C,'_i,2 if « > 1 and A^_i > 1 , 

(2.^j C,,Ai+i - j if ^ ^ ^ ^ f ^ ^,a;+i - ^.1 ■ 

We consider the set T{X) of fillings of A with entries in \n] which satisfy the following 
conditions: 

(1) the rows are weakly decreasing from left to right; 

(2) no column contains two entries a, b with a = ±6; 

(3) each column (with the exception of the leftmost one) is related to its left 
neighbor as indicated below; essentially, it differs from this neighbor by 
a "signed cycle", that is, a composition (ri, j) . . . (rp, j), where 1 < ri < 
. . . < r.p < j; furthermore, j varies from 1 to the length of the corresponding 
column, as we consider the columns from left to right. 

Here we let reflections in i?„ act on columns C like they do on signed permuta- 
tions; for instance, C{a, b) is the column obtained from C by transposing the entries 
in positions a, b and by changing their signs. Let us first explain the passage from 
some column dj to dj+i. There exist positions 1 < ri < . . . < rp < j (possibly 
p = 0) such that Cij+i differs from D = Cij(ri, j) . . . (rp, j) only in position j, 
while dj+iU) ^ {±Dir) : r G [A^] \ {j}} and C,,,+i(j) < Dij). To include the 
case j — \[ in this description, just replace Ci,j+i everywhere by Cij-|-i[l, A^] and 
use the conventions (|2.3p . Let us now explain the passage from some column C'^f. 
to C- j,^^. There exist positions 1 < ri < . . . < < fc (possibly p = 0) such that 
C[ — C-f.{ri,k) . . .{rp,k). This description includes the case k ~ A-, based on 
the conventions (|2.3p . 

Let us now define the content of a filling. For this purpose, we first associate 
with a filling a a "compressed" version of it, namely the filling a of the partition 
2A. This is defined as follows: 

(2.4) = C^' . . . C\ where T := C^^di , 

where the conventions (|2.3p are used again. Now define ct(cr) — (ci, . . . , c„), where 
Cj is half the difference between the number of occurences of the entries i and 7 in 
a. Sometimes, this vector is written in terms of the coordinate vectors e^, as 



(2.5) ct(cr) = ciEi + . . . + c„e„ = ^ X! 



here the last sum is over all boxes b of a, and we set e- :— —£i. 

We now define two statistics on fillings that will be used in our compressed 
formula for Hall-Littlewood polynomials. Intervals refer to the discrete set [n]. Let 



(2.6) aab 



1 if a, 6 > n 
otherwise . 



Given a sequence of integers w, we write for the subsequence w{i)w{i + 

1) . . ■w{j). We use the notation Nab{w) for the number of entries w{i) with a < 
w{i) < b. 

Given two columns _D, C of the same height d such that D > C component- 
wise, we will define two statistics N{D, C) and des(£', C) in some special cases, as 
specified below. 

Case 0. liD = C\ then N{D, C) := and des(i:>, C) := 0. 
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Case 1. Assume that C = D{r,j) with r < j. Let a := D{r) and b := D{j). In 
this case, we set 

NiD,C) ■.^Nj;^{D[r+l,j-l]) + \{b,a)\{±D{i) : i = 1, . . . , j}| + , 

and des(D, C) := 1 . 

Case 2. Assume that C = D{ri,j) . . . (rp, j) where 1 < ri < . . . < rp < j . Let 
:= D(ri, j) . . . (ri, j) for i = 0, . . . ,p, so that Dq = D and Dp = C. We define 

p 

N{D, C) N{D,^i,Di) , des(i?, C) := p . 

1=1 

Case 3. Assume that C difl'crs from D' := D{ri,j) . . . {rp,j) with 1 < ri < . . . < 
Tp < j (possibly p = 0) only in position j, while C{j) ^ {±D'{r) : r E [d] \ {j}} 
and C{j) < D'li). We define 

N{D,C) -.^ N{D,D') + Ncu),D'U){D[j + l,d]) , des{D,C) :=p+l. 

If the height of C is larger than the height d oi D (necessarily by 1), and 
N{D,C[l,d]) can be computed as above, we let N{D,C) := N{D,C[l,d]) and 
des(D, C) :— des(£', C[l, d\). Given a filling a in with columns Cm, ■ ■ ■ , Ci, 

we set 

m — 1 

N{a) ^7V(a+i,a)+£+(Ci); 

i=l 

here £+(Ci) is defined like in (|2.ip . Furthermore, we also set 

m — 1 

des((T) := ^ des(Q+i,Q) . 

i=l 

Note that des((T) essentially counts the descents in the rows of a. 

We can now state our new formula for the Hall-Littlewood polynomials of type 
C, which follows as a corollary of our main result, i.e.. Theorem 14.61 A completely 
similar formula in type B is discussed in Section \5\ We refer to Proposition 12.31 
and Remarks 14.71 for more insight into our formula. In particular, note that the 
Kashiwara-Nakashima tableaux of type C are, essentially, the surviving fillings in 
this formula when we set t = 0, and that, in some special cases, the statistic N(a) 
is completely similar to the Haglund-Haiman-Loehr inversion statistic (the more 
intricate of their two statistics). 

Theorem 2.1. Given a regular weight A, we have 

(2.7) Px{X; t)^ J2 ^^^"■^ (1 " ^)'^'''^'''' ' 

w/iere ^^'^^'■■■'^"^ :=x^V..a;^". 

Example 2.2. Consider the simplest case, namely n — 2 and A = (2, 1). This 
leads to considering fillings of the shape (3,2) with elements in [2], namely 



e 


c 


a 




d 


b 



The fillings need to satisfy the following conditions: 

(1) a < c < e, b < d; 

(2) a / ±6; 
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(3) either c — a and d ~ b, or c — b and d = a. 
For i e {1, 2}, let rii be half the difference between the number of i's and 7's in the 
multiset {a,b, c, d, e, e}. Given a proposition A, we let x(^) be 1 or 0, depending 
on the logical value of A being true or false. Then 

(a,&,c,d,e) 

It turns out that there are 27 terms in this sum, versus 70 terms in (Ram's version 
of) Schwer's formula. For instance, the terms contributing to the coefficient of X2 
correspond to the fillings 



ll 


1 


1 


|T 


2 


2 


[2 


2 


1 




2 


2 




1 


1 


7 


T 


2 



the associated polynomials in t are 

I-t, t{l~t) 

respectively. Note that these polynomials are obtained by compressing 3, 2, and 
2 terms in Schwer's formula, respectively. By symmetry, the coefficients of xi, X2, 
Xi^, and X2^ in P(2,i){xi, X2]t) are all {t + 2)(1 — t). Other fillings have an even 
larger number of terms in Schwer's formula corresponding to them, such as 



1 


2 


2 




1 


1 



which has 7; in other words, the associated polynomial in t, namely 1 — t, which 
contributes to the coefficient of the sum of 7 polynomials of the form 

t^{l — ty in Schwer's formula. In conclusion, we have 



P{2,l)ixi,X2;t) = xlx2 + Xix\ + xlx^ ^ + X1X2 + X^ ' X2 + X^ X2 

+ xl'^x^^ + (t + 2)(1 - t)[xi^ +X2+ x-^^ + x~ 



In order to relate our statistic N{a) to the Haglund-Haiman-Loehr inversion 
statistic, let us recall some definitions from [SI [T^] . We start by considering fillings 
T of the shape A with entries in [rl] , which are displayed in "Japanese style" , as a 
sequence of columns r — . . . Ci; here d = (Ci(l), . . . , Ci(A-)), so the entry in 
cell u = {i,j) is t(u) — Cj(i). Two cells u,v € X are said to attack each other if 
they are in one of the following two relative positions: 



An inversion of t is a pair of attacking cells (u, v) which have one of the following 
two relative positions, where a := t(u) < b := r(w): 



a 
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b 










b 




a 















The Haglund-Haiman-Loehr statistic inv(T) is defined as the number of inversions 
of T. The descent statistic, denoted des(r) (which is similar to des for fillings of 
A defined above, see below), is the number of cells u = {i,j) with j ^ 1 and 
t(u) > t{v), where w — {i,j~l). Now let, as usual, n(A) := and assume 
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that T has the following two properties: (i) t(u) ^ t(u) whenever u and v attack 
each other; (ii) r is weakly decreasing in rows. Then it was shown in |12j [Proposition 
2.12] that the so-called complementary inversion statistic cinv(T) := n(X) — inv(T) 
counts the triples of cells filled with a < b < c which have the following relative 
position (here the third cell might be outside the shape A, in which case we only 
require a < b): 





b 






c 


a 







Proposition 2.3. Let a in T(X) be a filling satisfying the following properties: (1) 
CI j^i — C[ j for all i and j — \, . . . , \[; (2) Q j+i only differs from Cij in position 
j . Let a be the filling of X given by 

■— Cai,iC'ai-i,i • • ■ Cii . 
Then N(a) — cinv(cr) and des((T) = des(CT). 

Proof. The equality des(cr) = des(CT) is clear, so we concentrate on the first one. 
Let m := Ai be the number of columns of A, and let Cm = Cmi, . . . , Ci = Cn 
be the columns of a, of lengths Cm '■= AJ„, . . . , ci := X'^; let := Cfe[l, Cfe+i], for 
k = 1, . . . , m — 1. We refer to a pair (i, j) with I <i < j < Ck and Ck{i) > Ck{j) 
as a (type A) inversion in Ck- It is easy to see that a satisfies the properties 
considered above: (i) cr(u) ^ (^{v) whenever u and v attack each other; (ii) a is 
weakly decreasing in rows. We start by evaluating N{C^Ck-^i^i) (see ^^). By 
definition, we have 

N{C'^Ck-Ui) - Nc,^^i^),C,(^)iCk[^+l,Ck]). 
1=1 

This is the number of inversions {i,j) in Ck for which Ck^i{i) < Ck{j)- If (*,i) 
is an inversion in Ck not satisfying the previous condition, then Ck-i{i) > Ck{j) 
(by property (i) of ct), and thus (i, j) is an inversion in C^_i (by property (ii) of 
a). Moreover, the only inversions of C^_]^ which do not arise in this way are those 
counted by the statistic cinv(CfcC^_]^), so 

NiC'Ck-ui) - e+iCk) - (e+iC'k^^) ~ cinviCkC'k^,)) . 
We conclude that 

m 

Nia) - i+{Ci) = J2 - + cmy{CkC'k_,) ■ 

k=2 

Now recall that A has no two parts identical. We clearly have c„j = 1 so ^+(C„i) = 0. 
Therefore, we have 

m 

N{a) = - e+iC'k-i) + cmv{CkC'k_,) = 

k=2 
m 

= ^ cinv(CfcCfc_i) = cinv(CT) . 

fe=2 

□ 



8 



CRISTIAN LENART 



3. Background on Ram's version of Sciiwer's formula 

We recall some background information on finite root systems and affine Weyl 
groups. 

3.1. Root systems. Let g be a complex semisimple Lie algebra, and I) a Cartan 
subalgebra, whose rank is r. Let $ C f)* be the corresponding irreducible root 
system, t)^ C t)* the real span of the roots, and C $ the set of positive roots. 
Let ai, . . . , Q!r G be the corresponding simple roots. We denote by ( • , • ) the 
nondegenerate scalar product on f)^ induced by the Killing form. Given a root a, 
we consider the corresponding coroot :~ 2a/ {a, a) and reflection s^. 

Let W be the corresponding Weyl group, whose Coxeter generators are denoted, 
as usual, by := Sa^. The length function on W is denoted by £{ ■ ). The Bruhat 
graph on W is the directed graph with edges u ^ w where w = usp for some 

(3 e and (.{w) > i{u); we usually label such an edge by /3 and write u w. 
The reverse Bruhat graph is obtained by reversing the directed edges above. The 
Bruhat order on W is the transitive closure of the relation corresponding to the 
Bruhat graph. 

The weight lattice A is given by 

(3.1) A := {A e f)R : (A, a"^) € Z for any a G 

The weight lattice A is generated by the fundamental weights wi,...,^^, which 
form the dual basis to the basis of simple coroots, i.e., (a;j,aj) = 6ij. The set A+ 
of dominant weights is given by 

A+ := {A G A : (A, a^) > for any a G $+}. 

The subgroup of W stabilizing a weight A is denoted by Wx , and the set of minimum 
coset representatives in W/Wx by W^. Let Z[A] be the group algebra of the weight 
lattice A, which has a Z-basis of formal exponents {x^ : A G A} with multiplication 

Given a G $ and k G we denote by Sa^k the reflection in the affine hyperplane 

(3.2) i7„,fc := {A G i}^ : (A,a^) = fc}. 

These reflections generate the affine Weyl group W^s for the dual root system 
:= {a^ : a G The hyperplanes Ha,k divide the real vector space 1)^ into 
open regions, called alcoves. The fundamental alcove Aa is given by 

Ao := {A G f)K : < (A, a^) < 1 for all a G $+}. 

3.2. Alcove walks. We say that two alcoves A and B are adjacent if they are 
distinct and have a common wall. Given a pair of adjacent alcoves A ^ B (i.e., 

having a common wall), we write A B if the common wall is of the form i?/3,fc 
and the root /3 e $ points in the direction from A to B. 

Definition 3.1. An alcove path is a sequence of alcoves such that any two consec- 
utive ones are adjacent. We say that an alcove path {Aq,Ai, . . . ,Am) is reduced if 
m is the minimal length of all alcove paths from Aq to A^ ■ 

We need the following generalization of alcove paths. 
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Definition 3.2. An alcove walk is a sequence il — {Aq, Fi, Ai, F2, . . . , Fm,Am,, Foo) 
such that Aq, . . . , Am are alcoves; Fi is a codimension one common face of the al- 
coves Ai-i and Ai, for i = 1, . . . , m; and F^o is a vertex of the last alcove Am- The 
weight F^o is called the weight of the alcove walk, and is denoted by fJ.{i~l). 

The folding operator (j)i is the operator which acts on an alcove walk by leaving 
its initial segment from Aq to Ai^i intact and by reflecting the remaining tail in 
the affine hyperplane containing the face Fi. In other words, we define 

<t>,{Sl) (Ao, Fi, Ai, . . . , A,_i, i^' = F,,A[,Fl+„A[^^, . . . , A'„,,F'^) ; 

here A'^ := Pi{Aj) for j e {i, . . . , to}, F'^ := Pi{Fj) for j e {i, . . . , m} U {00}, and pi 
is the affine reflection in the hyperplane containing Fi. Note that any two folding 
operators commute. An index j such that Aj-i = Aj is called a folding position 
of fi. Let fp{fl) := {ji < . . . < js} be the set of folding positions of If this set 
is empty, fl is called unfolded. Given this data, we define the operator "unfold", 
producing an unfolded alcove walk, by 

unfoid(r2) = . . . (t)j^ (n) . 

Definition 3.3. An alcove walk — {Aq, Fi, Ai, F2, . . . , Fm, Am, Foo) is called 
positively folded if, for any folding position j , the alcove Aj^i = Aj lies on the 
positive side of the affine hyperplane containing the face Fj . 

We now fix a dominant weight A and a reduced alcove path If := [Aq, Ai, . . . , Am) 
from Ao = Aq to its translate ylo + A = Am ■ Assume that we have 

Ao^A,J^...i^Am, 

where F :— (/3i, . . . ,(3m) is a sequence of positive roots. This sequence, which de- 
termines the alcove path, is called a X-chain (of roots). Two equivalent definitions 
of A-chains (in terms of reduced words in affine Weyl groups, and an interlacing 
condition) can be found in [T^ [Definition 5.4] and 15J [Definition 4.1 and Proposi- 
tion 4.4]; note that the A-chains considered in the mentioned papers are obtained 
by reversing the ones in the present paper. We also let := s/3. , and let be 
the affine reflection in the common wall of and Ai, for i = 1, . . . , to; in other 
words, fi := sp-^i-, where li := \{j < i : f3j = (3i}\ is the cardinality of the corre- 
sponding set. Given J — {ji < ... < js} Q [m] :— {1, . . . , m}, we define the Weyl 
group element 4'{J) and the weight p{J) by 

(3.3) (j}{J) := rj, . . . rj^ , fi{J) r^, . . . (A) . 

Given w £ W , we define the alcove path w{J\) :— {w{Aq),w{Ai), . . . ,w{Am))- 
Consider the set of alcove paths 

r{T) := {w(n) : w £ W^} . 

We identify any w(I\) with the obvious unfolded alcove walk of weight /i(w(n)) := 
w(A). Let us now consider the set of alcove walks 

3-+{Y') :— {positively folded alcove walks SI : unfold(ri) e VlT)} . 

We can encode an alcove walk in T+{r) by the pair {w,J) in X 2["1, where 

fp(r2) = J and unfold(17) = w{Ti) . 

Clearly, we can recover Vt from {w, J) with J ~ {ji < . . . < jg} by 
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Let A{T) be the image of J-'+iT) under the map il i-^ {w, J). We call a pair {w, J) 
in ^(r) an admissible pair, and the subset J C [m] in this pair a w- admissible 
subset. The following straightforward result is taken from [IJ] ■ 

Proposition 3.4. (1) We have 

(3.4) ^(r) = { [w, J)eW^x 2[™1 : J = {ji < . . . < j J , 

w > wrj-^ > . . . > wrj-^ . . . rj^ — w(f){J)} ; 

here the decreasing chain is in the Bruhat order on the Weyl group, its steps not 
being covers necessarily. 
(2) Ifn^ iw,J), then 

- u,(m(j)) . 

The formula for the Hall-Littlewood P-polynomials in [26] was rederived in [21] 
in a slightly different version, based on positively folded alcove walks. Based on 
Proposition 13. 4[ we now restate the latter formula in terms of admissible pairs. 

Theorem 3.5. [241 126j Given a dominant weight X, we have 

(3.5) Px{X;t)= t'('("')+'("'^(-'»-l^l)(l-t)l'^lx™(^(^». 

(iu,./)e-4(r) 

4. Specializing Ram's version of Schwer's formula to type C 

We now restrict ourselves to the root system of type C„. We can identify the 
space f)]^ with V :— R", the coordinate vectors being £i, . . . The root system 
$ can be represented as $ = {±ei ± Ej : i<i<j<n}U {±2ei : 1 < i < n}. The 
simple roots are ai = Si — Ei^i, for i = 1, . . . — 1 and a„ — 2e„. The fundamental 
weights are uji — ei + . . . + Si, for i — 1, . . . ,n. The weight lattice is A = Z". A 
dominant weight A = XiSi + . . . + A„_ie„_i + A„e„ is identified with the partition 
(Ai > A2 > . . . > A„_i > A„ > 0) of length at most n. A dominant weight is 
regular if all the previous inequalities are strict (i.e., the corresponding partition 
has all parts distinct and nonzero). We fix such a partition A for the remainder of 
this paper. 

The corresponding Weyl group W is the group of signed permutations Bn- For 
simplicity, we use the same notation for roots and the corresponding reflections, cf. 
Section [21 For instance, given 1 < i < j < n, we denote by {i,j) the positive root 
Ei — Ej, by {i,j) the positive root £i + Sj, and by {i,t) the positive root 2£,;. 

Let 

r(fc) :=r;,...r',ri(fc)...rfe(fc), 

where 

r;- ((i,j),(2,j),...,(j--i,j)), 

r,(fc):=((l,2)^ (2,|)^ (j-l,j), 
(j,fc + l), (j,fc + 2), (j,n), 
(jj), 

{j,n), (j,n-l), (j, fc + 1)). 

Lemma 4.1. T(k) is an LOk-chain. 

Proof. We use the criterion for A-chains in [15] [Definition 4.1, Proposition 4.4], cf. 
[15] [Proposition 10.2]. This criterion says that a chain of roots F is a A-chain if and 
only if it satisfies the following conditions: 
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(Rl) The number of occurrences of any positive root a in T is (A,Q!^). 

(R2) For each triple of positive roots {a, (3, 7) with 7^ = + the subse- 
quence of r consisting of a, /?, 7 is a concatenation of pairs (7, a) and (7, /3) 
(in any order). 

Letting X = ujk = £1 + ■ ■ ■ + £k, condition (Rl) is easily checked; for instance, a 
root (a, 5) appears twice in T(k) if a < 6 < fc, once if a < fc < 6, and zero times 
otherwise. For condition (R2), we use a case by case analysis, as follows, where 
a < b < c: 

(1) a = (a, b), P = (6, c), 7 = (a, c); 

(2) a = (a, 5), /3 = (6,c), 7 = (a,c); 

(3) a = (a,c), /3 = (6,c), 7 = (a, 6); 

(4) a = (&, c), /3 = (a,c), 7 = (a, fe); 

(5) a = (a, 6), /3 = (6, 6), 7 = (a, a); 

(6) a = {a,a), (3 = {b,b), 7 = (a, 6). 

Case (1) is the same as in type A. Each of the cases (2)-(4) has the following three 
subcases: k > c, b < k < c, and a < k < b, while each of the cases (5)- (6) has the 
following two subcases: k > b, and a < k < b. For instance, if 6 < fc < c in Case (3), 
then the subsequence of r(fc) consisting of a, /?, 7 is ((a, 6), (a, c), {a,b), {b,c)). □ 

Hence, we can construct a A-chain as a concatenation F := F'^^ . . .F^, where 

(4.1) F^ = F(A:) = T'^, . . .F^,F,i . . .F,,;,, , and F,, = F,(A:) , F^ = F^ . 

This A-chain is fixed for the remainder of this paper. Thus, we can replace the 
notation ^(F) with ^(A). 

Example 4.2. Consider n = 3 and A = (3,2, 1), for which we have the A-chain 
below. The factorization of F into subchains is indicated with vertical bars, while 
the double vertical bars separate the subchains corresponding to different columns. 
The underlined pairs are only relevant in Example 14.31 below. 

(4.2) F - F31 1 1 F'22F2iF22 1 1 F'l^F'igFnFisFig = 

= ((1,2),(1,3),(1,T),(1,3),(1,2)|| 

(1,2) I (1,3), (1,T), (1,3) I (1,2), (2,3), (2,2) , (2,3) || 

(1,2) I (1,3), (2, 3) I (1,1)1(1,2), (2, 2) I (1,3), (2, 3), (3,3)). 

We represent the Young diagram of A inside a broken 3x2 rectangle, as below. In 
this way, a reflection in F can be viewed as swapping entries and/or changing signs 
in the two parts of each column, or only the top part. 



1 


1 


1 




2 


2 






3 








3 


3 





Given the A-chain F above, in Section [5T^ we considered subsets J = {ji < . . . < 
js} of [to], where to is the length of F. Instead of J, it is now convenient to use 
the subsequence T of the roots in F whose positions are in J. This is viewed as a 
concatenation with distinguished factors and T/^, induced by the factorization 
gU) of F. 
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All the notions defined in terms of J are now redefined in terms of T. As 
such, from now on we will write 4>{T), IJ-{T), and |r|, the latter being the size 
of T, cf. (|3.3p . If J is a w-admissible subset for some w in i3„, we will also 
call the corresponding T a w-admissible sequence, and (w,T) an admissible pair. 
We will use the notation -4(r) and -4(A) accordingly. We denote by wT\-^^i . . . Tij 
and wTxj^^i ■ ■ - Tlf, the permutations obtained from w via right multiplication by 
the transpositions in Ta^^i, . . . , Tij and Tai.i? ■ ■ • , T'/fc, considered from left to right. 
This agrees with the above convention of using pairs to denote both roots and the 
corresponding reflections. As such, (/'(T) can now be written simply T. 

Example 4.3. We continue Example 14.21 by picking the admissible pair {w, J) 
with w = 1 2 3 S B3 and J = {2, 6, 12, 13} (see the underlined positions in (|4.2p ). 
Thus, we have 



T — T31 1 |T22T'2ir22 



7^i2Wi7^i2ri3 = ((1,3)11(1,2)1 1(2,2), (2,3)1 



The corresponding decreasing chain in Bruhat order is the following, where the 
swapped entries are shown in bold (we represent permutations as broken columns 
starting with w, as discussed in Example 14. 2[ and we display the splitting of the 
chain into subchains induced by the above splitting of T) : 





> 




2 




2 


3. 







3 




2 




2 




2 




2 




2 




2 




2 




2 




2 




2 


2. 


> 


3. 




3. 




3. 


> 


3^ 


> 


J_ 




1 




1 




1 




1 




1 




1 


1 








3 


1 


3 


1 


3 


1 


3 


1 


3 



m m m m a 

Given a (not necessarily admissible) pair {w, T), with T split into factors Ty and 



Tj'^ as above, we consider the permutations 



TTy = TTy (w,T) := wTai, 



ij-1 



T' 



when undefined, Tij^i and T/^,_]^ are given by conventions similar to (|2.3p . based 
on the corresponding factorization (|4.ip of the A-chain T. In particular, tta^^i = w. 

Definition 4.4. The filling map is the map f from pairs {w,T), not necessarily 
admissible, to fillings a — f{w,T) of the shape X, defined (based on the notation 



UM) by 

(4.3) a, =^,,[1,A:], Clfc = 7r^fc[l,A^] 

Example 4.5. Given (u>,T) as in Example 1 4. 3 [ we have 



f{w,T) 



ll 


3 


2 


2 


2 


2 


2 




2 


3 


3 


1 


1 


1 










3 


3 


3 



The following theorem describes the way in which our tableau formula (|2.7p is 
obtained by compressing Ram's version of Schwer's formula (j3.5p . 



w{p{T)). 



Theorem 4.6. (1) We have /(^(A)) = T{X). 

(2) Given any a G ^(A) and (w,T) G f~^{a), we have ct{f{w,T)) 

(3) T/ie following compression formula holds: 



(4.4) 



E 
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Proof. Wc start with part (1). The fact that f{A{X)) C J-{\) is clear from the 
definition of the set of fillings ^(A) in Section [2] and the construction (14. 1|) of the 
fixed A-chain F. Viceversa, given a filling a in J-{X), it is not hard to construct 
an admissible pair {w,T) in f~^{a). We will assign to the columns Cij and C'^j 
signed permutations pij and p'^j in _B„ recursively, starting with pu := Cn; in 
parallel, we construct the reverse rev(T) of the mentioned chain of roots T, and 
conclude by letting w := pai,i- Each time we pass to the left neighbor C'^f. of 
a column C'^ ^.^j^ = ^ifc(''i' k) . . . (r^. A;), we append to the part of rev(T) already 
constructed the roots (rp, fc), . . . , (ri, fc) and let p-^. := p- k) . . . (ri, fc). We 

proceed similarly when passing to the left neighbor C'ij of a column Cij+i, where 
Cij+i differs from D = Cy(ri,j) . . . (rp, j) only in position j; the only difference 
is that, in this case, we start by applying to Pij+i and appending to rev(r) the 
reflection which exchanges the entry Cij+i(j) with D(j), and then we proceed as 
above. 

Parts (2) and (3) of the theorem are proved in Section [6] and Section [71 respec- 
tively. □ 

Remarks 4.7. (1) The Kashiwara-Nakashima tableaux [5^ of shape A index the basis 
elements of the irreducible representation of sp2n of highest weight A. It is shown 
in Proposition 14.81 below that these tableaux correspond precisely to the surviving 
fillings in our formula (|2.7p when we set t = 0. 

(2) In (14. 4|) . in general, we cannot replace the filling map / with the map /, 
sending {w,T) to the compressed version f{w,T) of f{w,T). Indeed, consider 
n = 2, A = (3,2), and the following filling of 2A = (6,4), which happens to be the 
"doubled" version of a Kashiwara-Nakashima tableau: 



2|2 


2 


2 


1 


1 




1 


1 


2 


2 



If {w, T) E f ^((t), we must have w = 21 and 

Tcr2ir22- ((1,1)1(1,2), (2,2)), 

where the full A-chain factors as follows: 

r = r3i II r22-r2ir22 II ri2riiri2 ■ 

There are two elements (wjT^) and (wjT^) in / (ct), namely 

ri = ((l,2)), and t2 = ((1,T),(1,2),(2,2)). 

But we have 

j2 tWM+n^T)-\T\) _ ^^\T\ ^ - i) + (1 - 1)3 = (1 _ t)ii -t + e). 

In general, the above sum has several factors not of the form t or 1 — i, which are 
hard to control. 

(3) In order to measure the compression phenomenon, we define the compression 
factor c(A) like in [12 , as the ratio of the number of terms in Ram's version of 
Schwer's formula for A and the number of terms in the tableau formula. The 
compression factor is considerably larger in type C. For instance, for A = (3, 2, 1, 0) 
in C4 we have 23,495 terms in the compressed formula, while c(A) = 44.9. 
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Proposition 4.8. The map a ^ 'a (see (j2.4p ] is a bijection between the fillings 
a in J-(\) with N{a) — and the "doubled" versions of the type C Kashiwara- 
Nakashima tableaux of shape A. 

Proof. It was proved in pLj that for each type C Kashiwara-Nakashima tableau of 
shape A there is a unique admissible pair (w, T) whose associated chain in Bruhat 
order is saturated and ends at the identity, such that the compressed version W of 
a = f{w, T) is the "doubled" version of the given tableau. It follows that the term 
associated to (w,T) in ([33]) is - i)l^la;'"(^('^» , and therefore N{a) = 0, by 
(|4.4p . On the other hand, since P\{x;0) is the irreducible character indexed by A, 
which is expressed in terms of Kashiwara-Nakashima tableaux, we conclude that 
all a in J-{X) with N{a) — arise in this way. □ 

5. The tableau formula in type B 

We now restrict ourselves to the root system of type Bn- This can be represented 
as $ = {±£i ± £j : 1 < « < J < JT-} U {±£i : 1 < « < ri}. The simple roots are 
at = Si — £i+i, for i = 1, . . . , n — 1 and q:„ = £„. The fundamental weights are 
uJi — £i + . . . + £i, for i = 1, . . . , n — 1 and uin = ^(£1 + • ■ • + £n)- A dominant 
weight A — aiUi + . . . + a„a;„, where S Z>o, is identified with the partition 
fi ~ (ri"", . . . , 1"^); we let := ai + . . . + q:„, and write [l = {^i, . . . , ^J,e(p)). A 
dominant weight is regular if > for all i. Let us now fix such a weight A. 

The corresponding Weyl group W is the same group of signed permutations Bn 
considered above. For simplicity, we again use the same notation for roots and the 
corresponding reflections, cf. Section [S] The pairs {i,j) and have the same 
meaning as in type C, whereas (i) denotes the positive root Si. Note that, as a 
reflection in Bn, (i) is the same as {i,t) in type C. 

The canonical w^-chains and A-chains are very similar to those in type C. If 
k < n, let 

r(fc) ■.^r\...r',r,{k)...Tk{k), 

where 

r;.:=((i,j),(2,j),...,o--i,j),0-)), 

r,(fc) (2,j)^ 

(j,fc + l), (j,fc + 2), 
(j), 

{j,n), (j,n-l), (j, fc + 1)). 

On the other hand, we let 

T{n) :=r;...r;:=ri(n)...r„(n). 

Like in the type C case, we can prove that r(fc) is an cj^-chain for any k. Hence, 
we can construct a A-chain as a concatenation F :— F^'^^-' . . . F^, where F' = F(/ij). 
The filling map is defined like in Definition 14.41 This gives rise to fillings 

where each C is a concatenation of columns of height /i^, as follows: 

C := < Cil ■ ■ ■ \i i ^ 1 and pii = n 

[ Cii ' ' if i = 1 . 
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The fillings are subject to the same conditions (l)-(3) as in type C, where condition 
(3) is made more precise below. In fact, the above A-chain T specifies the way in 
which each column is related to its left neighbor. Essentially, everything is similar to 
type C, except for a small difference in the passage from some column C^'j, to C'^ f.^^. 
Namely, there exist positions 1 < ri < . . . < < fc (possibly p = Q) such that 
CU+i = C'iki^i^k) . . . irp,k), like in type C, or C^^^-^ = C^fc(ri, fc) . . . {rp,k){k), in 
which case we also require C- < n. 

The weight of a filling, and the statistics N{a) and des(tT) are defined completely 
similarly to type C. The only minor addition is the definition of N{D, C) and 
des(Z?, C) when C = D{ri, k) . . . {rp, k){k). With the notation in Case 2 of the 
definition of N{D,C) in Section [21 we set 

N{D, C) := N{D, Dp) + N{Dp, C) , des(£i, C) := p + 1 . 

Here N{D, Dp) is defined in Case 2 above, whereas N{Dp, C) is given by the second 
formula in (|7.ip ; more precisely, 

N{Dp,C) := ^\ia,a)\{±Dpit) : i = 1, . . . , fc}| , 

where a := Dpik). 

Given the above constructions, the proof of the following theorem is completely 
similar to its counterparts in type C, since no new situations arise. 

Theorem 5.1. Theorems 12.11 and 14.61 hold in type B, with the appropriate con- 
structions explained above. 

Remark 5.2. The situation in type D is slightly more complex. In this case, applying 
the above ideas leads to an analog of the compression formula (|4.4p which contains 
factors of the form 1 — t^ with fc > 1 in the right-hand side. However, these factors 
are not hard to control, while no extra factors appear. 

6. Proof of Theorem 14.61 (2) 

Recall the A-chain F in SectionlH Let us write F = (/3i , . . . , (3m) , as in Sect ion [3?2l 
As such, we recall the hyperplanes Hp^j,, and the corresponding affine reflections 

% = Sf3^,i^ = Sf3^ + IkPk- 

Now fix a signed permutation w in _B„ and a subset J = { ji < . . . < j s } of [m] 
(not necessarily w-admissible). Let H be the alcove path corresponding to F, and 
define the alcove walk Vl as in Section 13.21 by 

Given k in [m], let i = iik) be the largest index in [s] for which ji < k, and 
let 'jk ■= wrj^ . . .rj.{(3k). Then the hyperplane containing the face Fk of VL (cf. 
Definition 13. 2[) is of the form H^i^^„ik'i in other words 

Our first goal is to describe purely in terms of the filling associated to {w, J). 
Let tj. be the affine reflection in the hyperplane H^^ ^k- Note that 

tk = wfji . . . rjjkTj, . . . rj.w-^ . 

Thus, we can see that 

wrj, ...fj. =4 ...tj,w. 
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Let T = ((oi, ^i), ■ • • , (os, bs)) be the subsequence of T indexed by the positions 
in J, of. Section[4l Let T* be the initial segment of T with length i, let Wi :— wT^, 
and (Ti := /(w, T'), see (j2.4p . In particular, ctq is the filling with all entries in row i 
equal to and a := as — f{w, T). The columns of a filling of 2A are numbered 
left to right by 2Ai to 1. We split each segment F'"' of F into two parts: the head 
which is a concatenation of FJ.., and the tail which is a concatenation of F^., see 
(j4.ip . We say that the head corresponds to column 2k of the Young diagram 2 A, 
whereas the tail corresponds to column 2k — 1 (cf. the construction of f{w,T) 
in Section m and (12.4^ 1. If Pj^^^ = (0^+1,6^+1) = ia,,b) falls in the segment of F 
corresponding to column p of 2A, then (Ti+i is obtained from cr^ by replacing the 
entry Wi{a) with 'Wi(b) in the columns p — 1, . . . , 1 of (Ti, as well as, possibly, the 
entry Wi(b) with Wi{a) in the same columns. 

Now fix a position fc, and consider i = i(k) and the roots Pk, 7 ■— Jk, as above, 
where 7^ might be negative. Assume that /3fc falls in the segment of F corresponding 
to column q of 2A. Given a filling (j), we denote by (f)[p\ and (f)(j),q] the parts of (j) 
consisting of the columns 2Ai , 2Ai — 1, . . . , p and p—l,p — 2,...,q, respectively. We 
also recall the definition (|2.5p and conventions related to the content of a filling; 
this definition now applies to any filling of some Young diagram. 

Proposition 6.1. With the above notation, we have 

ruk = (ct(cr[g]),7'^) . 

Proof. We apply induction on z, which starts at i = 0, when the verification is 
straightforward. We will now proceed from ji < . . . < ji < k, where i = s or 
k < Ji+i, to ji < . . . < ji+i < fc, and we will freely use the notation above. Assume 
that Pjij^-^ falls in the segment of F corresponding to column p of 2A, where p > q. 
We need to compute 

where m = (ct(f7i[(7]), 7^), by induction. Let 7' := 7ji^.i, and tj^^^ = s^i^m', where 
m' = (ct((Ti[p]), (7')^): by induction. We will use the following formula: 

Thus, the proof is reduced to showing that 

m-m'(7',7^)-(ctK+i[g]),v(7^)). 

An easy calculation, based on the above information, shows that the latter equality 
is non-trivial only if p > g, in which case it is equivalent to 

(ctK+i(p,g]) -ct(a,(p,g]),7^) - (7',7^> (ct(a,+i (p, g]), (7')^) • 
This equality is a consequence of the fact that 

ct{ai+i{p,q]) = s^>{ct{ai{p,q])) , 
which follows from the construction of Ci+i from a.i explained above. □ 

Proof of Theorem H.Ql (2). We apply induction on the size of T, using freely the 
notation above. We prove the statement for T = , . . . , assuming it holds 

for T'' = iPj, , . . . , /3j J . We have 

wiKT) = wfji . . .r,-^^,(A) = tj^^, . . .tj.wiX) = i,-,+i(ct(CT^)) , 
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by induction. We need to show that 

(6.1) tj^^^{ct{as)) = ct(fT5+i) . 

Let 7 :— ^j^^-^ and assume that (3j^^-^ falls in the segment of F corresponding to 
column p of 2A. Based on Proposition [Ol (|6.ip is rewritten as 



(6.2) s^{ct{as)) + (ct(a4p]), 7^)7 = ct(a,+i) . 

Decomposing ct{as) as ct{as[p])+ct{(Ts{p, 1]) (cf. the notation above), and ct(CT5.+i) 
in a similar way, (j6.2p would follow from 

s^{ct{as[p])) + (ct(cr4p]),7^)7 = ct((T,+i[p]) , 
s^{ct{asip, 1])) = ct(crs+i(p, 1]) . 

The first equality is clear since as[p\ = (Ts+i[p\, while the second one follows from 
the construction of CTs-i-i from (7^ explained above. □ 

7. Proof of Theorem 14.61 (3) 

We start by recalling some basic facts about the group i?„ and some notation 
from Section [51 We will use the following notation related to a word w = wi . . .wi 
with integer letters, which is sometimes identified with its set of letters: 

w[i,j]:^Wi...Wj, Nabiw) ■.^\{a,b)nw\, Nab{±w) -.^ Nab{w) + Nabi-w) , 

where —w := WT . . . wj. Given words , . . . ^w^, we let 

Nabiw\ ...,WP):= Nab{w^) + ■■■+ NabiwP) . 

We also let 



Tab : = 



1 if a, 6 < n 
otherwise . 



With this notation, given a signed permutation w in i?„ and 1 < i < j < n, 
a :— w(i), b :~ w{j), we have the following facts: 

Nab{w[hj]) , 





£{w) 


- 1 


2 








£{w) 


- 1 


2 






i{w{i,j))- 


e{w) 


- 1 



(7.1) " '\ " ' = Naaiw[i, n]) , 

^abi^iJ - 1]; + 1."]) + Tab , 

assuming that the left-hand side is nonnegative (i.e., that we go up in Bruhat order 
upon applying the indicated reflection); these facts are used implicitly throughout. 

Given a chain of roots A, we define ^"^(A) like in p.4p except that we impose an 
increasing chain condition and w G W. The following simple lemma will be useful 
throughout, for splitting chains into subchains. 

Lemma 7.1. Consider (WjT) with T written as a concatenation Si . . . Sp. Let 
Wi := wSi . . . Si, so wq = w. Then 



l{i{w)+£{wT) -\T\) = ^{£(yjp^,)+i(wp) ~ \Sp\)+Y, l{£{w.-i)-£{w,) - \Sr\) ■ 
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Let A be the chain 

A:=((l,p+1), (l,p + 2), (l,n), 
(IJ), 



(l,n), (1,71-1), (1,P+1)). 

Proposition 7.2. Consider a signed permutation w in _B„ with a :— w{l), a 
position I < p < n, and a value b G {±a} U {±w[p + l,n]) such that b > a. Then 
we have 

(7,2) ^ t^WwT)-l(w)-\T\)^^ _^s^\T\ ^^N^t{wl2^p])^^ _^s^l-5^i . 

wT{l)=b 

here Sab is the Kronecker delta. 

Proof. Given s G {1, ±{p + 1), . . . , ±n}, we let Ag be the subchain of A starting 
with (l,s). We also let 

S{w,s):= J2 

T : (t«,T)e^''(Aa) 
wT(l}=b 



We consider three cases, depending on 6 = w{q), b = w{q), and b = a. The proof 
in the first case is identical to that of the analogous result for type A, namely 
[H] [Proposition 5.3]. 

Case 2. Let c := w{q) ~ b, and p < q < s. We start by showing 

(7.3) Siw,s) = t'^M^l2,<]-i]Mq+hs],±w[s+i,n])+T,,(^^ _ ^^ 

We use induction on s, which starts at s = g. For s > q, let :— w[2,q — 1], 
:— w[q + 1, s — 1], := w[s + 1, n], and d := w{s). The sum S'(w, s) splits into 
two sums, depending on (l,s) ^ T, and (l,s) G T. By induction, the first sum is 

Again by induction, if a < d < c, then the second sum is 

^A^„3(^,l™^±«,^)+r„,(^ „ t)S{w{l,s),7^) 

otherwise, it is empty. Adding up the two sums into which S{w,s) splits, we obtain 
The last claim rests on the easily verified facts that, ii a < d < c, then 

Tad + T-^, = Tac , N -^c) + N-^{a) = Nac{d) . 

Still assuming that c — w{q) = b and p < q, we now show that 

(7.4) Siw, T) = t'^a-dH2,g-l]Mci+hn]}+rL (1 - t) , 

where 



1 if a < c < n 
otherwise . 



Let := w[2,q — 1], like before, and := w[q + l,n]. The sum S{w, 1) sphts 
into two sums, depending on (1, 1) ^ T, and (1, 1) G T. By (|7.3p . the first sum is 

^(w,n) = t^"^^'"' (1-i). 
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Again by (17. 3p . if c < a < n, then the second sum is 

t^^^(^"-^"){l - t)Siw{l,T),n) = _ . 

otherwise, it is empty. Adding up the two sums into which S{w,s) sphts, we obtain 

Assuming that c ^ w{q) — b and p < q < s, we now show that 

(7.5) S{w, s) = iJV„^(«'[2,9-l],'"[9+l.«-l])+<.(l _ t) _ 

We use decreasing induction on s. Like before, we let := w[2,q — 1], w'^ := 
w[q + 1, s — 1], and d :— w{s). The sum S'(w, s) sphts into two sums, depending on 
(1, s) ^ T, and (1, s) £ T. By induction, the first sum is 

S{w, S + 1) - t^"^(«''.«'''i)+<c(l _ t) . 

Again by induction, if a < d < c, then the second sum is 

t^^-ii^'--^"^ (1 - t)S{w{l, s),S + l)^ ^N^,{n,^cn,-)+N^^{n,\n,-a)+r;^ _ . 

otherwise, it is empty. (In both calculations, induction works by substituting 1 for 
n + 1 when s — and by using (|7.4p in this case.) Adding up the two sums into 
which S{w, s) splits, we obtain 

The last claim rests on the easily verified fact that, if a < d < c, then 

Nad{c) + = ^ac • 

Case 3. Let us now assume that b = a. We need to show that 

(7.6) Siw,p+1)^ t^a^M2,p]) (1 _ t) . 

We use decreasing induction on p, which starts at p = n; in this case A only contains 
the pair (1, 1), so the above convention of substituting 1 for n + 1 works well here 
too. For p < n, we let d := w{p + 1). The sum S{w,p + 1) splits into two sums, 
depending on (l,p + 1) ^ T, and (l,p + 1) G T. By induction, the first sum is 
S'(u.,p + 2) = t^"^('"[2,p]rf)(^„^)^ 

If a < d < a, then by (|7.5p of Case 2, the second sum is 

^N^,(w[2,p])^^ „ t)S{w(l,p+ l)),p + 2) = iJV„,(™[2,p])+JV,,(»[2,p])+rL(i _ i)2 . 

otherwise, it is empty. Adding up the two sums into which S{w,p + 1) splits, we 
obtain the desired result. 

Case 2 (continued). Assuming that c — w{q) — b and p < q, we now show that 

(7.7) S{w, q) = t^-^C-P.^^i]) (1 _ t) . 

The sum S{w, q) splits into two sums, depending on (1, q) ^ T, and (1, q) £ T. By 
(|7.5p of Case 2, the first sum is 

S{w, g + 1) = t^rMH2.q-i])+rL (1 - t) . 
If a < c < n, then by (|7.6p of Case 3, the second sum is 

tN^^i-^l^^1-^\){^l _ t)S{w{l, q)),q+l)= ^Af.c(-[2,?-l])+JV.,(^„[2,g]) _ ^)2 . 

otherwise, it is empty. Adding up the two sums into which ^(w, q) splits, we obtain 
the desired result. 
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The final step in Case 2 is to prove that 
(7.8) S{w,p+ 1) = t^a-.M^,p])^i _ 

This can be done by decreasing induction on p, starting with p = q — 1, which is 
the case proved in (|7.7p . The procedure is completely similar to the ones above, 
and, in fact, to the one for type A in [12 [Proposition 5.3]. □ 

Let us consider the chain 

$:=ri(n)...r„(n) = ((l,l), 

(7.9) 

(l,n), (2,n), {n-l,n)). 

We denote by the subchain of $ starting with Given a signed permutation 

w, recall the definition (|2.ip of ^+(w) and £-{w). Given with 1 < i < j < n, 
we also define 



(7.10) ei{w) ■.= \{{k,l) : (fc,I)G$\$„ , w(fc)>ii;(/)}|, 

tl ■.^£-iw)-£'l{w). 

Proposition 7.3. Fix with I < i < j < n and a signed permutation w in Bn- 
We have 

(7.11) ^ ^^{l{w)+HwT)-\T\) i^-y _ ^•^\T\ ^ _^l+(w)+tj^(w) _ 
T: (u;,T)e^(*.j) 

In particular, if the above sum is over {w,T) £ A{^), then the right-hand side is 

Proof. Let us denote the sum in the left-hand side of (|7.1ip by S{w,i,j), and the 
corresponding sum over (w, T) G A{^ij \ {(«, j)}) by S'{w, i,j)- We view the chain 
$ as a total order on the pairs (i, j), with (1,1) being the smallest pair. With this in 
mind, we use decreasing induction on pairs («, j). Given such a pair, if w{i) < w{j) 
then the induction step is clear, so assume the contrary. We can now split S{w, i,j) 
into two sums, depending on (i, j) ^ T and (i, j) G T. By induction, the first sum 
is 

By induction and Lemma l7.H the second sum is 

(l-t)t^W-)-^(«'(^j))"l)5'(«;(z,j),i,j) 
= (1 - t)t^('^("')-^(«'(»j))-l)+^+('"(*j))+^-("'(*j)) . 
The induction step is completed once we show that 

e+H+i'liw) = i {£{w) - £{w{i,j)) - 1) +f+(u;(*,j)) +fi(w(z,j)). 
The latter equality can be rewritten as 

A£+{w) + Afi(u)) - 1 = Afi{w) , 

where A£+{w) := £+{w) — £+{w{i,j)), and similarly for the other two variations. In 
order to prove this, let us first fix k between i and j, and analyze the contribution 
to the three variations of the pairs {i,k) and (fc,j), cf. (|2.ip and (|7.10p . For 
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simplicity, let a w{i), b := w{k), and c := where a > c. The mentioned 

nonzero contributions are as follows: 

(1) the pair (i, k) contributes 1 to A£+{w) if a > 6 > c; 

(2) the pair (fc, j) contributes —1 to Ai?_|_(u)) if a < < c, which is equivalent 
to a >b > c; 

(3) the pair (j, k) contributes 1 to Atl (w) if a >b > c; 

(4) the pair {k,j) contributes 1 to At^{w) if a > 6 > c. 

Note that the second and third contributions cancel out, whereas the first one is 
equal to the fourth one. The analysis is completely similar if fc < z or fc > j. The 
pair only contributes 1 to A.tl(w). As far as the pairs and are 

concerned, the contribution of the first one to Atl_ (w) and of the second one to 

□ 



A£_(w) are both equal to aac, see ()2.6 



Proof of Theorem \4:.6\ (3). Fix a filling a in J^(A) with columns Cij and C'^j, as 
explained in Section[51 Recall the chain $ := Ti{n) . . . r„(n) = Fn . . . Fi„ in (|7.9p . 
By splitting the A-chain F into the tail (f> and its complement, and by using Lemma 
17.11 the sum in the left-hand side of (14. 4p can be rewritten as 



(7.12) 



( \ 



^ ^^(Uw)-HwT)-\T\) ^-y _ ^^\T\ 

\ Tii = ...=Ti„=0 



E 

kT: {Cii.T)£A('1>) 



t2 



i(<'(Cii)+^(Ciir)-|T|) 



(l-t) 



T\ 



here the column Cn, which has height n, is viewed as a signed permutation in 
By Proposition [731 the second bracket is 

In order to evaluate the first bracket, we will reverse all chains. Let us start by 
recalling the construction (|4.ip of the A-chain F, and in particular the order in which 
the subchains F^ and T[j are concatenated (including the conventions in Section [2] 
related to F^^+i and T[ j_^-f^). We denote by F[^- and (F^^ )'' the corresponding reverse 
chains. Also recall that we defined A^i ■) like in p.4p except that we imposed an 
increasing chain condition and w dW. We consider pairs {wij, Sij) in ^''(F[^) and 
{w[pS[j) in A~{{T^jY), where wij and w[j are defined by 



Wi 



■ S, 



where the concatenation order for Sij ci^^ 
particular, w'^ y = Cn. Given this notation, we define the sum 



and S"', comes from that for Fy and F^^ ; 



WijSij[l,X'i]=Cij 
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and the similar sum E^^-. We can now evaluate the first bracket in the right-hand 
side of (jTl^ : 

E (1 - tr\ = . . . . . . s,- . . . e;,,, . 

Tii = ...=Ti„=0 

In fact, we first write the sum in the left-hand side as an iterated sum, which factors 
in the way shown above because E^j only depends on Wjj[l,A^] = Ci.j+i[l,A^] 
(rather than the whole Wij), by Proposition [721 

We conclude the proof by calculating the sum Ei j , the calculation for E^^ being 
similar but simpler. For simplicity, let d := A^, w = Wy, C := Qj+i [1, A^], and 
D := Cij. Assume that C differs from D' := D{ri,j) . . .{rp,j) with 1 < ri < . . . < 
'^p < j (possibly p = 0) only in position j. Let T^j = AA', where 

A:=((j,d+1), {j,d + 2), (j,n), 

{3,n), {j,n-l), {j,d+l)), 

A' :=((j-l,j),...,(2,j),(l,j)). 
Correspondingly, the chains Sij split into a head 5, which can vary, and a fixed tail 

•5" :== ((fp,j), ■ ■ • , {rij)) ■ 

We have 

wS{3)=D'U) 

where e := l{£{wSS') - £{wS) - p). By Proposition [7?1 the sum in the right-hand 
side is 

t'^c(j).D'(j)iD[j+i,d])(^l_ty^ 

note that this sum is missing when D' = C, which is another possibility. The 
exponent e is calculated based on (|7.ip . □ 
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